MEMORANDUM November 28, 2018 


TO: Annie Wolfe 
Officer, Secondary Curriculum and Development 


FROM: Carla Stevens 
Assistant Superintendent, Research and Accountability 


SUBJECT: | STRATEGIC READING AND WRITING (SRW) REPORT, 2017-2018 


Strategic Reading and Writing (SRW) is a reading and writing intervention course available to 
sixth- through tenth-grade students who did not meet Grade Level standards on the State of 
Texas Assessments of Academic Readiness (STAAR) grades 3-8 reading the previous school 
year. Students in this sample consisted of sixth- through eighth-grade students who did not 
meet Grade Level standard on the 2017 STAAR 3-8 reading exam and who had a score on the 
2018 STAAR 3-8 reading test. The evaluation used a teacher’s survey, classroom 
walkthrough, and a paired t-test to determine the implementation, teacher perceptions and 
experiences, and student outcomes. 


Key findings include: 

e All SRW survey respondents were certified and trained with 94.1 percent having at least 
three-years teaching experience and 88.2 percent having three or more years’ experience 
conducting reading interventions. Most respondents participated in the SRW professional 
development and between 57.1 and 71.4 percent attended monthly support meetings. 

e Respondents either agreed or strongly agreed that they assisted their students with small- 
group instruction around texts, monitored students’ reading and writing, checked their 
students’ understanding and gave “just-in-time” scaffolding, organized students into needs- 
based intervention groups, and that their students could work independently on reading and 
writing tasks at their independent levels. 

e Respondents agreed or strongly agreed that they modeled the writing process for their 
students, supported their students though small-group instruction and collaborative writing 
experiences, and scheduled with students to provide writing feedback. 

e Most respondents used Vocabulary.com and Achieve 3000 digital resources in the delivery 
of SRW but most never used Mindplay, MackinVia, Digital Magazines, and Imagine 
Learning. Most somewhat agreed or agreed that Saddleback “Hi-Low” texts were 
accessible, that available texts were of a high quality, and that they had access to 
classroom libraries. 

e SRW appeared to result in statistically significant gains on the 2018 STAAR 3-8 reading 
performance of sixth-grade (mean difference = 49.4 scale score points (Ssp)), seventh- 
grade (mean difference = 150.2 ssp), and eighth-grade (mean difference = 188.7 ssp) 
students. 


e SRW had small to large effects sizes on the sixth-grade (d = 0.31, one-third of one standard 
deviation increase); seventh-grade (d = 0.98, almost one standard deviation increase); and 
eighth-grade (d = 1.20, more than one standard deviation increase) students’ 2018 STAAR 
3-8 reading performance. 

e Using Fryer (2012) estimates, the SRW course resulted in an extra four, twelve, and fifteen 
months of schooling for sixth, seventh, and eighth-grade students, respectively. 

e Non-economically disadvantaged, non-at-risk, G/T, and non-LEP students appeared to have 
benefited most from exposure to SRW across all three middle-school grade levels. 


Further distribution of this report is at your discretion. Should you have any further questions, 
please contact me at 713-556-6700. 


Attachment 
cc: Noelia Longoria 


Mechiel Rozas 
Jessica Chevalier 
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HISD STRATEGIC READING AND WRITING: 
IMPLEMENTATION, TEACHER PERCEPTIONS AND 
EXPERIENCES, AND STUDENT OUTCOMES, 2017-2018 


Executive Summary 


Strategic Reading and Writing (SRW) is a reading and writing intervention course available to sixth- through 
tenth-grade students in the Houston Independent School District (HISD) who did not meet Grade Level 
standards on the State of Texas Assessments of Academic Readiness (STAAR) grades 3-8 reading the 
previous school year or who fell below the 40" percentile on the District’s norm referenced test (NRT) total 
reading score (HISD, 2017). SRW classes are restricted to 20 students and are facilitated by certified and 
trained teachers. The objective of SRW is to: (1) meet students at their instructional level to remediate and 
strengthen literacy systems within, about, and beyond texts, (2) prepare students for grade-level 
coursework through aligned genre-specific tasks and support, and (3) prepare students to increase 
achievement on state tests through strategic actions and increased reading levels. 


The purpose of this evaluation was to assess the SRW effect on students’ reading performance. The 
evaluation also assessed teachers’ perceptions and experiences with the implementation of SRW. An 
online survey using SurveyMonkey™ was administered to teachers who completed the SRW professional 
development. Students were linked to their Public Education Information Management System (PEIMS) 
data, SRW teachers, and their 2017 and 2018 STAAR 3-8 reading test results. Only students with both 
STAAR scores and who did not meet Grade Level standard on the 2017 STAAR 3-8 reading were included 
in the sample. A repeated-measures design was used to determine students’ reading gains and the effect 
of SRW on students’ reading by the end of the 2017-2018 academic year. 


Key findings 


e ~All SRW survey respondents were certified and trained with 94.1 percent having at least three-years of 
teaching experience, while 88.2 percent had three or more years of experience in conducting reading 
interventions. Most respondents participated in the SRW professional development and between 57.1 
and 71.4 percent attended the monthly support meetings. 


e Respondents either agreed or strongly agreed that they assisted their students in small-group 
instruction around texts, monitored their students’ reading and writing, checked their students’ 
understanding, gave “just-in-time” scaffolding, organized students into needs-based intervention 
groups, and that their students could work independently on reading and writing tasks at their 
independent levels. 


e Respondents agreed or strongly agreed that they modeled the writing process for their students, 
supported their students though small-group instruction and collaborative writing experiences, and 
scheduled with students to provide writing feedback. 


e Most respondents used Vocabulary.com and Achieve 3000 digital resources in the delivery of SRW but 
most never used Mindplay, MackinVia, Digital Magazines and Imagine Learning. Most somewhat 
agreed or agreed that Saddleback “Hi-Low” texts were accessible, that available texts were of a high 
quality, and that they had access to classroom libraries. 
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SRW resulted in statistically-significant gains on the 2018 STAAR 3-8 reading performance of sixth- 
grade (mean difference = 49.4 scale score points (Ssp)), seventh grade (mean difference = 150.2 ssp), 
and eighth-grade (mean difference = 188.7 ssp) students. 


SRW had a small to large effect size on sixth-grade (d = 0.31), seventh-grade (d = 0.98), and eighth- 
grade (d = 1.20) students’ 2018 STAAR 3-8 reading performance. 


These effect sizes translated into an additional four, twelve, and fifteen months of schooling at the sixth, 
seventh, and eighth-grades, respectively, using Fryers (2012, p. 9) estimate of 0.08 standard 
deviations as equivalent to one extra month of schooling. 


Non-economically disadvantaged, non-at-risk, G/T, and non-LEP students appeared to benefit most 
from exposure to SRW across all three middle-school grade levels. 


Recommendations 


Based on the impact of SRW on students’ reading performance, its implementation should be continued 
and where possible strengthened to meet the needs of academically-disadvantaged students in the 
district. SRW summer professional development (PD) with monthly support meetings reflect the U.S. 
Department of Education What Works Clearinghouse best practice on teacher professional 
development and should be maintained, strengthened, and replicated. 


The SRW should continue to provide support through its monthly meetings and to also use this as an 
opportunity to monitor and address any issues that may surface during implementation. 


Given the effectiveness of the SRW intervention, the strategies should be used routinely in HISD 
classrooms to prevent reading failure and underperformance and improve students’ reading and 
reading comprehension. 


Future evaluations should look at the impacts of multiple years of SRW exposure on student’s academic 
performance in STAAR 3-8 reading. 


Given respondents’ feedback, SRW managers should look with favor into providing additional training 
on the use of Saddleback “Hi-Low” books, developing lesson plans for the upper high school grades, 
and providing additional SRW resource materials, among others. 


SRW managers, school administrators, and teachers should explore creative ways to ensure teachers 
and students utilize the SRW digital resources available on the Hub to enhance their instruction, student 
engagement and student learning, for example, inter-class and inter-grade use competitions. 


Based on respondents’ feedback, exemplary SRW instruction and practices should be identified, 
archived, and made accessible to SRW teachers in response to their request for flexible learning 
groups, best SRW research-based strategies, and activities with deeper student engagement and not 
just access to open educational resources (OER) questions. 
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Introduction 


Strategic Reading and Writing (SRW) involves the implementation of reading and writing intervention 
strategies for underperforming sixth- to tenth-grade students in the Houston Independent School District 
(HISD). It is offered as a course designed to provide academic interventions for students who have fallen 
below the 40* percentile on the district level literacy assessments and state tests (HISD, 2017). SRW is 
given in addition to the English language arts (ELA) core curriculum time requirement. 


SRW is designed to: (1) meet students at their instructional level to remediate and strengthen literacy 
systems within, about, and beyond texts, (2) prepare students for grade-level coursework through aligned 
genre-specific tasks and support, and (3) prepare students to increase achievement on state tests through 
strategic actions and increased reading levels. SRW students work independently on reading and writing 
tasks at their independent levels and meet with teachers in small groups around texts at their instructional 
level. The small-group meetings are designed to check for assumptions and provide students with just-in- 
time scaffolds, simultaneously. Only foundation classes are taught using a whole-group approach and to 
equip students with instruction, practice, and feedback for their independent work. “Hi-Low” text libraries 
are provided for all campuses involved in SRW and are rotated among SRW teachers. In addition, students 
have access to classroom libraries in all core content classrooms, and teachers use comprehension toolkits 
for texts and for additional strategic lessons. SRW classes are restricted to 20 students or less. 


Reading and writing are complex skills that have implications for students’ learning, academic performance, 
and achievement. Based on Accelerated Reader database, Renaissance Learning Inc. (2018) reported that 
American high school students were reading books with readability levels between 6.5 in 2014—2015 and 
7.2 in 2016-2017, which fall short of expected future readability level of 13.8 (first-year college students’ 
level). The report reviewed the reading habits of 9.4 million K-12 grade students using 323 million books 
and non-fiction articles from 30,062 schools in all 50 states and Washington, DC (Renaisance Learning 
Inc., 2018). The report also showed that 9-12'-grade students read an average 4.8 to 6.0 books during the 
2016-2017 academic year (Renaisance Learning Inc., 2018). However, this was only one database. 
Students could have had access to other reading sources, including school and public libraries. Results of 
the 2018 National Assessment of Educational Progress (NAEP) revealed that 37 percent (2017) of fourth 
grade, 36 percent (2017) of eight-grade and 37 percent (2015) of twelfth-grade students were assessed as 
at or above proficient readers (McFarland, et al., 2018). 


The purpose of this evaluation was to assess the implementation of SRW, SRW teachers’ perceptions and 
experiences with its implementation, and the reading outcomes of students who were exposed to the 
strategies. The evaluation sought to answer the following questions: 


1. What were the perceptions and experiences of teachers who implemented SRW interventions in their 
classrooms? 


2. What was the demographic and educational composition of the student sample used in this study? 


3. How much reading gains did SRW students in HISD experience during the 2017-2018 school year? 
Literature review 


Intervention research for struggling fourth- to twelfth-grade readers has had a long history (Scammacca, et 
al., 2016). Much has changed in the conceptualization of reading interventions and the methods used to 
determine their effectiveness in improving reading results for readers who struggle (Scammacca, et al., 
2016). Vaughn, et al. (2015) conducted a study using a randomized control trial of ninth- and tenth-grade 
students with significant reading problems. The 50-minute small-group intervention, which was 
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administered once per day over two-years resulted in significant gains in reading comprehension (effect 
size = 0.43) when compared to students in the sample who were exposed to business as usual. The 
improved reading among students in the treatment group resulted in higher social studies scores (Vaughn, 
et al., 2015). This is an indication that improvement in reading and comprehension results in performance 
improvements in other curriculum areas. 


Seventy-five students who were exposed to close reading intervention showed improvement in reading 
achievement, attendance, and self-perception when compared to 247 of their peers who received traditional 
intervention. The close reading of complex texts involved annotations, repeated reading, text-dependent 
questions, and discussions (Fisher & Frey, 2014). Close reading, peer collaboration, and wide reading of 
young adult literature made the difference (Fisher & Frey, 2014). 


A quasi-experimental study involving 17 eighth-grade students investigated the impact of a repeated 
reading fluency intervention with prosody, counterbalanced with a reading strategies intervention (Van Wig, 
2016). The study was designed to promote feelings of achievement due to participation in activities intended 
to promote reading success. Students were divided into A-days and B-days and their results compared 
using ANOVA with a repeated-measures design. Results indicate that fluency intervention focused on 
prosody was effective in improving prosodic ability, but inconclusive on fluency components of rate and 
accuracy improvement (Van Wig, 2016). Students improved their perceptions of their reading ability and 
behavior after participation in the reading intervention (Van Wig, 2016). 


Teacher efficacy and implementation of a supplemental intervention for struggling adolescent readers were 
compared to determine their effect on student achievement. Sixth- and ninth-grade teachers were 
compared. Sixth-grade teachers had higher levels of efficacy while ninth-grade teachers had higher levels 
of implementation (Cantrell, Almasi, Carter, & Rintamaa, 2013). Teacher efficacy was positively related to 
students’ reading comprehension and overall reading achievement. Implementation was positively related 
to students’ growth in vocabulary (Cantrell, Almasi, Carter, & Rintamaa, 2013). How teachers think about 
their instructional abilities have implications for students’ performance. 


Students entering secondary schools with reading difficulties were exposed to 20 weeks of reading 
intervention delivered by trained teaching assistants (Clarke, Paul, Smith, Snowling, & Hulme, 2017). The 
three 35-minute sessions involved 287 students, ages 11—13, from 27 schools, who were randomly 
assigned into three groups: reading intervention targeting word recognition and decoding skills, reading 
intervention plus comprehension, or a waiting list control group (Clarke, et al., 2017). The interventions did 
not produce statistically-significant gains in word reading. The reading intervention plus comprehension 
produced significant gains in reading comprehension (d = 0.29) and vocabulary (d = 0.34) (Clarke, et al., 
2017). 


A review of experimental studies on the effects of reading interventions administered to grades K-12 
English learners (EL) published between 2000 and 2012 found significant moderate to large effects (effect 
sizes (ES) of 0.47—2.34) for reading or listening comprehension in five of the twelve studies reviewed 
(Richards-Tutor, Baker, Gersten, Baker, & Smith, 2016). Interventions included explicit instruction. Ten of 
the programs used published intervention programs. Variables such as group size, intervention minutes, 
and type of personnel delivering the intervention were not significant predictors of reading outcomes 
(Richards-Tutor, et al., 2016). 


Another meta-analysis of the effect of reading and writing was conducted using 54 experimental and quasi- 
experimental studies involving 5,018 kindergarten- to twelfth-grade students to determine if reading 
interventions improved student writing (Graham, et al., 2018). As predicted, teaching reading strengthened 
writing, resulting in statistically-significant effects for overall writing measures (ES = 0.57) and specific 
measures of writing quality (ES = 0.63), words written (ES = 0.37), or spelling (ES = 0.56). The effects on 
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writing was maintained over time (ES = 0.37) (Graham, et al., 2018). Having students read or observe 
others interact with text enhanced writing performance with a statistically-significant impact on an overall 
measure of writing (ES = 0.36) and specific measures of writing quality (ES = 0.44) or spelling (ES = 0.28) 
(Graham, et al., 2018). 


Method 


Relevant teachers underwent initial summer training and preparation for the implementation of SRW. In 
addition, teachers attended six monthly follow-up meetings during the academic year. A digital record of 
teachers who attended these follow-up meetings was not maintained, limiting their use in the analysis of 
their effects on students’ reading performance. Students whose teachers attended the initial professional 
development (PD) sessions and who did not meet standard on the 2017 STAAR 3-8 reading made up the 
study population. Reading was used because students were assigned to the SRW who did not meet 
standard on the 2017 STAAR 3-8 reading tests. A repeated-measures designed determined the effect of 
SRW on students’ reading one year after the intervention. The repeated-measures design is a robust test 
that allows for control on all variables by using the performance of the same sample at time 1 and time 2 
and comparing the means. The difference in means is used as the determinant of performance and the 
basis for calculating program effects. Students with both 2017 and 2018 STAAR 3-8 reading scale scores 
were included in the sample. These were 732 sixth-grade, 744 seventh-grade, and 577 eighth-grade 
students. 


Teacher PD data, downloaded from HISD eLearning database and students’ demographics and 
educational data taken from the PEIMS, were linked using unique identifiers. Students’ demographic and 
educational data included race and ethnicity, gender, as well as at-risk, economic, special education, 
limited-English proficiency (LEP), and gifted and talented (G/T) status. These were linked to students’ 
STAAR data sing unique identifiers as well. 


A web-based survey using SurveyMonkey™ was administered to teachers who attended the professional 
development workshop in preparation for implementing SRW. They were surveyed on their SRW 
perceptions and experiences with SRW professional development, support, and instructional practices as 
well as students’ reading and writing practices. The survey was developed with assistance from the two 
teacher development specialists (TDS) who were involved in the implementation oversight of the SRW 
initiative. They reviewed and revised the survey items to ensure adequate coverage and validity. They 
tested the survey before final transmission to participants, which was then emailed to 70 teacher 
participants with a reminder to complete the survey two weeks later. A total of 18 participants competed the 
survey. This is a response rate of 25.7 percent which falls within the expected response rate for online 
surveys (Resnick, 2012). Summary of responses were collated by SurveyMonkey™ using percentages or 
weighted averages where Likert scales were used. Survey statements were rated on a scale of 1—5, with 
one being the lowest and 5 being the highest. Survey results were presented in figures, tables, and graphs. 
Open-ended questions were analyzed using emergent themes. 


Ten classrooms were randomly selected for walkthrough visits based on the list of teachers who attended 
the SRW professional development workshop. Emails were sent out to principals of the selected 
classrooms to solicit their participation in those visits. A reminder was sent out two weeks later. Two 
principals responded. An ethnographic approach was taken involving observations and the use of field 
notes to record key aspects of the SRW lessons. The key Texas Essential Knowledge and Skills (TEKS) 
covered in the lesson were identified. The TEKS to be covered were outlined in the SRW scope and 
sequence document developed by the HISD Secondary Curriculum and Development Department. 


Data were analyzed using paired t-test on the reading scores of SRW students who did not meet the 2017 
STAAR 3-8 Approaches Grade Level reading standards. Students’ 2017 scale scores were used as the 
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pre-score and their 2018 scores were used as the post-score to determine any significant changes in 
performance. Cohen’s d was used to determine the effects of SRW on the performance of students. Fryer’s 
(2012) estimates of 0.08 standard deviation as equivalent to one extra month of schooling was also used 
to determine SRW effect. The data were subject to and satisfied normality, collinearity, and homoscedastic 
testing using the Kologorov-Sminov statistic, and Normal and Detrended Q-Q plots. 


Limitations 


e Given HISD’s focus on improving literacy, districtwide, students in the sample may have been exposed 
to other reading initiatives and programs and their extraneous effects on SRW performance. Linking 
SRW teachers to their students was an attempt to reduce or control the effects. 


e With few schools participating in the walkthroughs, it was inappropriate to make credible and valid 
statements about the implementation fidelity of the SRW course. 


e Monthly meeting attendance was not digitally recorded and, therefore, unavailable for use in the 
analyses. The summer PD with monthly follow-up represented best practice in professional 
development and its impact on students’ performance (Yoon, Duncan, Lee, Scarloss, & Shapley, 
2009). 


e Teachers may have had more than one year of SRW professional development. While this was not 
considered in the evaluation, it is likely, that that may have had positive effects on students’ 
performance. It is not always possible to control for all the extraneous variables that affect a treatment 
or an outcome (Huber, 2015). 


Results 


What were the perceptions and experiences of teachers who implemented SRW interventions in 
their classrooms? 


Figure 1 through Figure 10 summarized the survey results from 18 teacher respondents of teacher 
perceptions and experiences related to SRW professional development, their requisite skills, instructional 
practices, and assessment of their students’ strategic reading and writing activities. Table A-1 through 
Table A-10, Appendix A, (pp. 23-25) provide details. Calculations may differ from 100 percent due to 
rounding. 


Figure 1. Percent of SRW Survey Respondents Who Were Trained and Certified, 2017-2018 (n = 17) 


100.0 


Percentage 


0.0 


Yes No 
Trained and Certified 


e All survey respondents revealed that they were trained and certified in the delivery of SRW. Table 
1A, Appendix A (p. 23) provides details. 
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Figure 2. Years of Teaching Experience of SRW Survey Respondents, 2017-2018 (n = 17) 


Percentage 


100.0 
90.0 
80.0 
70.0 
60.0 


50.0 


40.0 29.4 35.3 


30.0 17.7 
20.0 11.8 
a o om 
0.0 a 
0-2 years 3-5 years 6-10 years 11-15 years 16 -20 years 20 or more years 
Years of Teaching 


Note: Numbers may total greater than 100 percent due to rounding. 


Overall, 94.2 percent of the SRW survey respondents had at least 3 years of teaching experience. 
Table A-2, Appendix A (p. 23) provides details. 


About 64.8 percent of the SRW survey respondents had at least 11 years of teaching experience. 
Most SRW teacher respondents (35.3%) had 20 or more years of teaching experience, followed by 


those with 3—5 years of experience (29.4%), followed by those with 11-15 years and 16-20 years (17.7 
and 11.8 percent, respectively). 


Figure 3. HISD SRW Respondents’ Experiences with Conducting Reading Interventions, 2017- 
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e = Overall, 88.3 percent of SRW survey respondents had at least 3 years of experience conducting 
reading interventions. Table 3-A, Appendix A (p. 23) provides details. 


e About 58.9 percent of respondents had 6 or more years of experience conducting reading 
interventions 
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Most survey respondents had either 3-5 years (29.4%) or 6-10 years (23.5%) of experience in 
conducting reading interventions. 


About 17.7 percent of SRW survey respondents had 16-20 years of experience conducting reading 
interventions. 


Figure 4. Number of SRW Classes Being Taught by Survey Respondents, 2017-2018 (n = 16) 


Percenage 


100.0 


90.0 
80.0 
70.0 
60.0 
50.0 
40.0 
30.0 


20.0 
10.0 11.8 
0.0 


1to2 3 to 4 5 or more Other (please specify) 
Years 


About 64.7 percent of SRW survey respondents taught up to four SRW classes during the 2017- 
2018 school year. Table A-4, Appendix A (p. 23) provides details. 


Most SRW respondents (35.3%) taught between one to two SRW classes during the 2017-2018 
school year, followed by 29.4 percent who taught three to four classes. 


Figure 5 shows the weighted average of SRW survey respondents’ responses to four professional 
development statements. Respondents were asked to indicate using “No” or “Yes” weighted on a Likert 
Scale of 1 to 2, respectively. Table A-5, Appendix A (p. 24) provides details. 


Figure 5. Survey Respondents’ Participation in SRW Professional Development Training, 2017- 
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e = Slightly more SRW respondents answered “No” than answered “Yes” to the statement “The information 
| learned during the one-day “job-alike’” training prepared me to begin the implementation of SRW.” This 
statement received an average rating of 1.44 out of 2.00. 


e Most SRW respondents confirmed that they attended the “job-alike” training for the 2015-2016, 2016— 
2017, and 2017-2018 academic year, with increasing weighted averages of 1.57, 1.67, and 1.73, 
respectively. 


Figure 6 displays the survey respondents’ attendance at the monthly SRW meetings from the 2015-2016 
to the 2017-2018 academic years. Table A-6, Appendix A, (p. 24) provides details. 


Figure 6. Survey Respondents’ Participation in Monthly SRW Meetings, 2017-2018 


70.0 
60.0 
50.0 
40.0 
20.0 : 6 6 6 
10.0 21.4 21.4 
0.0 


| attended SRW | attended SRW | attended the | attended the | attended the | attended the | attended the 
monthly meetings monthly meetings SRW meeting on SRW meeting on SRW meeting on SRW meeting on SRW meeting on 
during the 2015- during the 2016- September12, October 12, 2017 November 9, 2017 January 11, 2018 February 8, 2018 


Percentage 
wo 
oO 
oO 


2016 school year. 2017 school year. 2017 on on dos and don't on reading on Vocabulary - on Authentic 
establishing of small group conference - Word Works. Assessment and 
classroom instruction. progress Student Choice 
routines. monitoring with (Project-Based 
use of authentic Learning) 
"Just Right" texts. 
=No mYes 


e About 78.6 percent of survey respondents attended SRW monthly meetings during the 2015—2016 and 
2016-2017 academic years. 


e Most survey respondents, between 57.1 and 71.4 percent, attended SRW monthly meetings during the 
2017-2018 academic year. 


e Except for September and October 2017, over 70 percent of respondents attended the monthly SRW 
meetings for the 2017—2018 academic year. 


e The initial September meeting for the 2017-2018 academic year had the lowest monthly attendance 
rate for survey respondents (57.1%). 


Survey respondents were asked to rate their agreement or disagreement with teacher efficacy, resources, 
and instructional practices’ statements on a scale of 1 to 5, with 1 being “strongly disagree,” 2 being 
“disagree,” 3 being “somewhat agree,” 4 being “agree,” and 5 being “strongly agree.” Figure 7 presents the 
results. Table A-7, Appendix A (p. 24) provides details. 
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Figure 7. Survey Respondents Agreement or Disagreement With Selected SRW Efficacy and 
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On average, respondents agreed or somewhat agreed (weighted average of 3.64 on a 5.0 rating) that, 
compared to a year ago, they had a deeper understanding of what it meant to support students in need 
of reading and writing interventions. 


On average, respondent agreed that their students worked independently on reading and writing tasks 
(weighted average of 3.92 of 5.0). 


Respondents agreed or strongly agreed that they met with their students in small groups around text 
at their instructional levels (weighted average of 4.38 of 5.0); that they checked for their students’ 
understanding and gave “just-in-time” scaffolds (weighted average of 4.46 of 5.0); that they monitor 
students’ reading and writing performance (weighted average of 4.77 of 5.0); and that they organize 
their students into needs-based intervention groups (weighted average of 4.62 of 5.0). 


On average, respondents disagreed or somewhat agreed that they had access to Saddleback “Hi-Low” 
texts on their campuses (weighted average of 2.69 of 5.0) or somewhat agreed that Saddleback “Hi- 
Low” texts available to their classrooms were of high quality (weighted average of 3.00 of 5.0). 


Respondents somewhat agreed that they had access to classroom libraries in all core content-based 
areas (weighted average of 3.31 of 5.0). 


Figure 8 is a radial graph, which shows respondents’ assessment of their HISD students’ reading practices. 
Respondents were asked to rate their level of disagreement or agreement with key SRW statements related 
to students’ reading practices on a scale of 1 to 5 as depicted for Figure 7. Table A-8, Appendix A (p. 25) 
provides details. 
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Figure 8. Radial Graph Showing the Weighted Average of SRW Respondents’ Assessment of Their 
HISD Students’ Reading Practices, 2017-2018 
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e Respondents somewhat agreed that their students could (1) self-monitor their reading and make 
adjustments using rereading and questioning (weighted average of 3.31 of 5.0), (2) that their students 
could summarize, paraphrase, and synthesize texts in ways that make meaning and logical order 
(weighted average of 3.31 of 5.0), and (3) their students could reflect understanding to monitor their 
reading comprehension (weighted average of 3.31 of 5.0). 


e On average, respondents somewhat agreed or agreed (weighted average 3.46 of 5.0) that (1) their 
students could monitor and adjust their comprehension using background knowledge, rereading, and 
visualizing and (2) their students used prior knowledge to comprehend text (weighted average of 3.77 
of 5.0). They agreed that they were able to schedule and confer with their students to provide feedback 
about their reading (weighted average of 4.00 of 5.0). 


e On average, respondents somewhat agreed or agreed (weighted average 3.62 of 5.0) that their 
students made inferences such as drawing conclusions and making generalizations and predictions. 


e On average, respondents somewhat agreed or agreed (weighted average of 3.58 of 5.0) that their 
students could establish purposes for reading selected texts based on their desire to enhance 
comprehension. 


Figure 9 displays survey respondents’ assessments of their writing instructional practices and their 
students’ writing practices. Respondents were asked to rate their agreement or disagreement with key 
statements about their writing instructional practices and students’ writing practices on a scale of 1 to 5, 
with 1 being “strongly disagree,” 2 being “disagree,” 3 being “somewhat agree,” 4 being “agree,” and 5 
being “strongly agree.” Table A9, Appendix A (p. 25) provides details. 
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Figure 9. SRW Respondents’ Assessment of Their Writing Instructional Practices, 2017-2018 
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Writing Practices 


e On average, respondents agreed or strongly agreed that (1) they modeled the writing process for 
their students (weighted average of 4.31 of 5.0), (2) they supported their students through small 
group instruction and collaborative writing experiences to meet their needs (weighted average of 
4.08 of 5.0), and (3) they scheduled and conferred with students to provide feedback about their 
writing (weighted average of 4.25 of 5.0). 


e Onaverage, respondent somewhat agreed or agreed that (1) their students’ writing styles reflected 
a variety of purposes (weighted average of 3.54 of 5.0) and (2) that they used professional learning 
conference (PLC) time to analyze their students’ writing samples (weighted average of 3.42 of 5.0). 


e On average, respondents somewhat agreed that (1) their students were leading their own writing 
conferences (weighted average of 3.00 of 5.0) and (2) their students were selecting and publishing 
pieces of work in their own writing portfolios (weighted average of 3.17 of 5.0). 


Survey respondents were asked to assess their use of selected digital resources in SRW. Respondents 
were asked to share the frequency with which they used these resources in their SRW instruction or 
practices on a scale of 1—5 with 1 being “never,” 2 being “rarely,” 3 being “sometimes,” 4 being “often,” and 
5 being “very often.” Figure 10 shows the respondents’ use of selected digital resources. Table A-10, 
Appendix A (p. 25) provides details. 
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Figure 10. Respondents’ Use of Digital Resources in SRW, 2017-2018 
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Digital Resources 


On average, survey respondents have either “never” to “sometimes” used Mindplay (weighted average 
of 1.15 of 5.0), MackinVia (1.23 of 5.0), Digital Magazines (weighted average of 2.14 of 5.0), and 
Imagine Learning (weighted average of 2.23 of 5.0) during SRW instruction. 


Most respondents said they used Achieve 3000 (3.43 of 5.0) and Vocabulary.com (weighted average 
of 3.21 of 5.0) “sometimes” to “very often” (3.2 of 5.0) to deliver SRW. 


Just as many respondents said they “very often” used SRW Hub Master Course as said they never 
used it (weighted average of 2.69 of 5.0) during SRW instruction. 


Most respondents said that they used Renaissance Universal Screener “often” (weighted average of 
3.14 of 5.0) in their delivery of SRW. 


Six respondents indicated that they used other resources for SRW like, PBS, Quizlet, Language Live, 
and Newselac or that they did not have access to Mindplay, Imagine Learning, and MackinvVia, that the 
technology was unavailable on their campuses, or was not available at all. 


Survey respondents were asked to identify any professional development they would like to receive to 
further support SRW instruction. Fourteen respondents provided suggestions. Five respondents said none, 
not sure, or not applicable. Two respondents recommended “scaffolding/scaffolding writing.” The other 
suggestions included: 


Activities with a deeper engagement not just open education resources (OER) questions, 
Having lesson plans developed for upper grades of high school, 

How to access ready-to-use resources, 

Training on Saddleback texts, and 


How to create flexible grouping with best research strategies. 
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Respondents were also asked to provide additional details or information they would like to see in the HISD 
Curriculum Planning Guides to support SRW instruction. There were nine responses. Two respondents 
said none or not applicable. The following were recommended: 


e Activities like Document-Based Question (DBQ) that can be used as a social studies resource, 

e Better design of reading master courses so that they are better aligned to the English master course, 
e Ready-to-use resources and mentor test at the middle school level, 

e Access to Study Island, and 


e Lessons should include the various reading programs. 


Finally, respondents were asked for additional comments related to SRW. There were six responses 
including: 


e Access to all tests for diagnosis and remediation, 
e Lessons in HUB lack rigor and do not support IB programs, 
e Please add Study Island, and 


e Need better training in Renaissance Universal Screener. 
Classroom Observation 


Two schools responded to the walkthrough requests to observe one class each. Only one lesson was 
observed. The other class did not deliver SRW instruction during the visit but, instead, focused on year-end 
school activities. Observations of the first lesson was conducted on May 17, 2018 and began at 11:30 am. 
An ethnographic approach involving observation and the use of field notes to record key aspects of the 
lesson delivery was used. 


The teacher introduced students to a Domestic Incident Report (DIR), to be completed, based on an article 
entitled “Lamb to the Slaughter.” Students were expected to complete the DIR form based on their reading 
of the article. The teacher paced the class through the reading. There were two clearly written and visible 
lesson objectives: 


1. 1 will increase reading comprehension and use text to support my thinking in written response. 


2. | can speak with my table group to clarify and show my understanding of the assignment. 


Students, however, were not seated by table groups. Using the DIR forms, students were asked to identify 
the information they needed to complete the task: date, race of characters, relationship between characters, 
contacts, suspect, victim, and history. Students were asked to identify their role, which was to prepare the 
victim’s statement, who was killed in the story, by role-playing the victim. Students, therefore, had to use 
information from the article, their experiences, perceptions, and imagination to determine the facts of the 
case. Students read the article along with the teacher who asked questions to identify evidentiary details 
using several comprehension strategies throughout the lesson. 


e Teacher asked students to determine what they already knew about the characters based on the 
reading to establish the context for the story. 
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e She checked for understanding by asking for meaning of key words in the story like irony, routine, 
and doting. 


e The teacher asked students for evidence from the story to support claims, positions, and 
conclusions. 


e She drew on students’ personal experiences related to domestic incidents to identify the sequence 
of events. 


e The teacher used literary devices like inference based on evidence from the reading, for example 
to determine the motive and the weapon used. 


e The teacher removed the ending of the story to encourage students to determine how they thought 
the story ended. 


e She used extensive probes to determine the preponderance of the evidence and conclusion of the 
story. 


e Students used the title of the article, and other elements in the story to determine how the victim 
died and who the perpetrator was. 


During the observations, the lesson content was matched against the Secondary Curriculum and 
Development, 2017-2018 Scope and Sequence document, Strategic Reading and Writing, Reading 1: 
Grade 9 (6 Cycle, Unit 6, Informational text). The Scope and Sequence document outlines the SRW- 
related Texas Essential Knowledge and Skills/Students’ Expectations (TEKS/SEs). Students were able to 
cover 13 of the 20 TEKS/SEs for reading in Unit 6. These TEKS/SEs included: 


e RE.9.3.E Reading for enjoyment; 

e RE:9.4.A Use prior knowledge and experience to comprehend; 

e RE.9.4.D Summarize text by identifying main ideas and relevant detail; 
e RE9.5.C Support inferences with text evidence and experience; 


e RE.9.5.C Analyze the presentation of information and the strength and quality of the evidence used by 
the author; and 


e RE.9.5.E Distinguish facts from simple assertions and opinions. 
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What was the demographic and educational composition of the student sample used in this study? 


Figure 11. Demographic and Educational Composition of the Student Evaluation Sample, 2017-2018 
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e The sample comprised 732 sixth-grade, 744 seventh-grade, and 577 eighth-grade students. 


e Most students in the sample were economically disadvantaged (75.0-83.2%), at-risk (81.0%—82.4%), 
and Hispanic (69.7—74.0%), or black (22.2—27.6%). 


e The percentage of students in the sample who received special education services ranged from 5.1 to 
12.2 percent. G/T students in the sample ranged from 11.7 to 18.3 percent while LEP students ranged 
from 27.8 to 39.9 percent. 


How much reading gains did SRW students in HISD experience during the 2017-2018 school year? 


Figure 12 reports the pre- and post-SRW performance of sixth- through eighth-grade students using their 
2017 and 2018 STAAR 3-8 reading results. Table B-1 through Table B-3, Appendix B (p. 26) provide 
details. 


Figure 12. Paired T-Test Results for SRW Sixth- Through Eighth-Grade Students, 2017-2018 
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e Sixth-grade students who were exposed to SRW had a post-scale score (M = 1504.5, SD = 131.9) that 
was higher than their pre-scale score (M = 1455.1, SD = 75.6). The difference was statistically 
significant, t(731) = 8.5, p. < .001. The Cohen’s d statistic (0.31) indicated a small effect size. 


e Seventh-grade students who were exposed to SRW had a post-scale score (M = 1578.3, SD = 127.4) 
that was higher than their pre-scale score (M = 1428.1, SD = 78.6) on the 2018 STAAR 3-8 reading. 
The difference was statistically significant, t(743) = 26.7 p. < .001. The Cohen’s d statistic (0.98) 
indicated a large effect size. 


e Eighth-grade students who were exposed to SRW had a post-scale score (M = 1623.3, SD = 134.4) 
that was higher than their pre-scale score (M = 1434.6, SD = 73.4). The difference was statistically 
significant, t(576) = 28.8 p. < .001. The Cohen’s d statistic (1.20) indicated a very large effect size. 


e These effect sizes translated, on average, to an additional four, twelve, and fifteen months of schooling 
at the sixth, seventh, and eighth-grades, respectively, using Fryers (2012, p. 9) estimate of 0.08 
standard deviations as being equivalent to one extra month of schooling. 


Figure 13 shows the percentage of SRW students in the sixth-, seventh-, and eighth-grades who met Grade 
Level standard on the 2018 STAAR 3-8 reading assessments. SRW students did not meet Grade Level 
standards the previous year, 2017. 


Figure 13. Percentage of SRW Students by Grade who met the Approaches Grade Level Standard 
on the 2018 STAAR 3-8 Reading 
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e Based on Figure 13, 42.8 percent of SRW sixth-grade students performed at or above the Approaches 
Grade Level standard on the 2018 STAAR 3-8 reading assessment. Almost one-fifth of the SRW 
students (17.7%) performed at or above the Meets Grade Level standard and 7.2 percent performed 
at the Masters Grade Level standard. 


e According to Figure 13, 53.4 percent of SRW seventh-grade students performed at or above the 
Approaches Grade Level standard on the 2018 STAAR 3-8 reading assessment. About one-fifth 
(20.1%) of students performed at or above the Meets Grade Level standard, and 8.8 percent performed 
at or above Masters Grade Level standard. 
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e More than 58 percent of SRW eighth-grade students performed at or above the Approaches Grade 
Level standard on the 2018 STAAR 3-8 reading assessment. Almost 30 percent (27.9%) performed at 
or above the Meets Grade Level standard, and 13.4 percent performed at or above the Masters Grade 
Level standard. 


e Figure 14 displays the percentage of SRW students disaggregated by grade and select demographic 
and educational attributes who performed at or above the Approaches Grade Level standard on the 
2018 STAAR 3-8 reading assessments. Table B-4, Appendix B (p. 27) provides details. 


Figure 14, Percentage of SRW Students Disaggregated by Key Variables and Grade Who 
Performed at or Above the Approaches Grade Level Standard on the 2018 STAAR 3-8 
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e A higher proportion of SRW female students compared to male students performed at or above the 
Approaches Grade Levels for the sixth-, seventh-, and eighth-grade students on the 2018 STAAR 
reading exams. 


e More than 50 percent of seventh- and eighth-grade SRW economically-disadvantaged students 
performed at or above the Approaches Grade Level standard on the 2018 STAAR reading tests. They 
were out performed, however, by their counterparts who were not economically disadvantaged. 


e Almost 50 percent of seventh- and eighth-grade SRW at-risk students performed at or above the 
Approaches Grade Level standard on the 2018 STAAR 3-8 reading tests. However, they were 
outperformed by their peers who were not at-risk. 


e More than three-quarters of sixth-, seventh-, and eighth-grade SRW G/T students performed at or 
above the Approaches Grade Level standard on the 2018 STAAR 3-8 reading exams. Almost 50 
percent of seventh-grade (49.4%) and more than 50 percent of eighth-grade (53.5%) SRW non-G/T 
students performed at or above the Approaches Grade Level standard. 


e §=©Sixth-grade SRW Black students (46.8%) outperformed their Asian, Hispanic, and White peers on the 
2018 STAAR 3-8 reading test. SRW Asian students (62.5%) outperformed their Black, Hispanic, and 
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White peers at the seventh grade, and SRW White students (100%) outperformed their Asian, Black, 
and Hispanic peers at the eighth grade on the 2018 STAAR 3-8 reading test. 


Discussion 


This report used a teachers’ survey, classroom observations, and paired t-test to measure the impact of 
SRW on the STAAR 3-8 reading performance of middle school student. Students in the sample did not 
meet Grade Level standards on the 2017 STAAR 3-8 reading and were exposed to SRW course as 
intervention during the 2017-2018 school year. Students were linked to their teachers who were exposed 
to summer and monthly SRW professional development in preparation for and support of the delivery of 
SRW in their classrooms. 


Results of the survey indicated that all teacher respondents had the basic requisites for delivering SRW in 
their classrooms. It also showed that they may have been well prepared and supported during the 
implementation. They were all certified and trained to deliver reading interventions, as well. In addition, 
most respondents (94.1%) had at last three years of teaching experience, while 88.2 percent had at least 
three years of experience with conducting interventions. Moreover, most of the respondent participated in 
the profession development activities (weighted average of 1.73 of 2.0) for the 2017-2018, with most of 
them having participated during the previous two years (weighted average of 1.67 and 1.57 of 2.0) as well. 
Between 57.1 and 71.4 percent of these respondents attended the monthly SRW support meeting. On 
average, respondents agreed (weighted average 3.64 of 5.0) that they had a deeper understanding of what 
it meant to support students in need of reading and writing intervention. 


On average, respondents either agreed or strongly agreed with statements regarding their SRW 
instructional practices associated with assisting their students with small group instruction around texts, 
monitoring students’ reading and writing, checking their students’ understanding, and giving “just-in-time” 
scaffolding. Respondents agreed or strongly agreed (weighted average 4.62 of 5.0) that they organize their 
students into needs-based intervention groups, and that their students could work independently on reading 
and writing tasks at their independent levels (weighted average of 3.92 of 5.0). Notwithstanding, 
respondents wanted to know how to access ready-to-use SRW resources as well as access to training on 
Saddleback texts. 


Respondents were not as confident about the material resources that were available to them in support of 
SRW. With a weighted average between 2.69 and 3.31 of 5.0, respondents somewhat agreed or agreed 
(1) that they had access to Saddleback books, (2) that Saddleback books were of high quality, and (3) that 
they had access to classroom libraries. This is not to suggest that other materials were unavailable. Most 
respondents said they used Vocabulary.com and Achieve 3000 (3.21 and 3.43 of 5.0, respectively). 
However, most respondents said they never used digital resources like Mindplay, MackinVia, Digital 
Magazines, and Imagine Learning (weighted average of 1.15, 1.23, 2.14, and 2.3 of 5.0, respectively) that 
supported SRW. 


On average, respondents somewhat agreed or agreed with statements about their students’ reading skills. 
Of the ten statements related to students’ writing skills, nine had weighted averages between 3.31 and 3.77 
of 5.0. Respondents agreed (weighted average of 4.0 of 5.0) that they conferred with their students to 
provide reading feedback. Average responses were not as strong for writing. With weighted average of 
3.00—3.54 of 5.0, respondents somewhat agreed or agreed that students were leading their own writing 
conferences, selecting and publishing pieces of work in their writing portfolios, and that students’ writing 
tasks reflected a variety of purposes. 


Respondents tended to be confident in the SRW writing instructional practices. Respondents, therefore, 
agreed or strongly agreed that they modeled the writing process for their student, supported their students 


HISD Research and Accountability 19 


STRATEGIC READING AND WRITING REPORT, 2017-2018 


through small-group instruction and collaborative writing experiences, and scheduled with students to 
provide writing feedback (weighted average of 4.08—4.31 of 5.0). Yet respondents were not as confident 
about their students’ writing practices or activities (weighted average of 3.00 to 3.54). Close monitoring 
allows teachers to know what their students do or can do. Teacher efficacy correlated positively, with 
students’ reading comprehension and overall reading achievement in a study involving supplemental 
intervention for struggling sixth through ninth-grade adolescent readers (Cantrell, Almasi, Carter, & 
Rintamaa, 2013). 


SRW appeared to have a significant effect on the reading performance of sixth- through eighth-grade HISD 
students in the sample whose teachers underwent SRW professional development. SRW had a small to 
large effect size (ES) of 0.31, 0.98 and 1.20 on the sixth-grade, seventh-grade, and eighth-grade, 
respectively, STAAR 3-8 reading. These substantive gains can be translated as four, twelve and fifteen 
additional months of schooling, respectively because of the strategies (Fryer, 2012). The importance of 
reading strategies in improving reading comprehension and performance is well-documented (Manarin, 
2012). Van Wig’s (2016) study of eight-grade students, using a similar repeated measures analyses, found 
improved perceptions of student's own reading abilities and behavior after being exposed to reading 
intervention strategies. Fish and Frey (2014) also recoded improved reading achievement for students who 
were exposed to close reading interventions. 


While all students in the sample performed below Grade Level standards in 2017, the analyses showed 
that 42.8, 53.4 and 58.1 percent performed at or above Grade Level standards at the sixth, seventh, and 
eighth-grades, respectively, in 2018. When disaggregated by demographic and education subgroups, 
between 54.8 and 96.5 percent of non-economically-disadvantaged students, non-at-risk, G/T, and non- 
LEP students performed at or above the Approaches Grade Level standards across all three middle school 
grades. While disadvantaged students (economically disadvantaged, at-risk, LEP, and student receiving 
special education services) did not do nearly as well, between 5.8 and 54.8 percent performed at or above 
the Approaches Grade Level standard on 2018 STAAR 3-8 reading at the sixth, seventh, and eighth 
grades. In addition, between 17.7 and 27.9 of middle school students in the sample performed at above the 
Meet Grade Level standard, and of these, between 7.2 and 13.4 percent performed at or above the Masters 
Grade Level standard. 


Overall, SRW appeared to be effective in improving middle school students’ reading performance after one 
year of reading intervention. One key reason for the impact may be the professional development, followed 
by yearlong support through monthly meetings as recommended in the research literature (Yoon, Duncan, 
Lee, Scarloss, & Shapley, 2009). Observation of one classroom, although it cannot be inferred to all SRW 
classrooms, does provide some indication of what may have happened during the delivery of SRW. 
Additionally, most SRW teachers were trained, certified, and had at least three years of experience in 
conducting interventions. 


Recommendations 


e Based on the impact of SRW on students’ reading performance, its implementation should be continued 
and where possible strengthened to meet the needs of academically-disadvantaged students in the 
district. SRW summer PD with monthly support meetings reflect the U.S. Department of Education 
What Works Clearinghouse best practice on teacher professional development and should be 
maintained, strengthened, and replicated. 


e The SRW should continue to provide support through its monthly meetings and to also use this as an 
opportunity to monitor and address any issues that may surface during implementation. 
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e Given the effectiveness of the SRW intervention, the strategies should be used routinely in HISD 
classrooms to prevent reading failure and underperformance and improve students’ reading and 
reading comprehension. 


e Future evaluations should look at the impacts of multiple years of SRW exposure on student’s academic 
performance in STAAR 3-8 reading. 


e Given respondents’ feedback, SRW managers should look with favor into providing additional training 
on the use of Saddleback “Hi-Low” books, developing lesson plans for the upper high school grades, 
and providing additional SRW resource materials, among others. 


e SRW managers, school administrators, and teachers should explore creative ways to ensure teachers 
and students utilize the SRW digital resources available on the Hub to enhance their instruction, student 
engagement, and student learning, for example, inter-class and inter-grade use competitions. 


e Based on respondents’ feedback, exemplary SRW instruction and practices should be identified, 
archived, and made accessible to SRW teachers in response to their request for flexible learning 
groups, best SRW research-based strategies, and activities with deeper student engagement and not 
just access to open educational resources (OER) questions. 
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Appendix A: Survey Responses 


Table A-1. Proportion of HISD SRW Teacher Respondents Who Were Certified or Trained in 


English Language Arts, 2017-2018 


Yes 100.0 17 
No 0.0 0 

Total 17 
No responses 1 


Table A-2. Years of Teaching Experience for HISD SRW Respondents, 2017-2018 


Years 7 Responses 

0 n 
0-2 years 5.9 1 
3-5 years 29.4 5 
6-10 years 0.0 0 
11-15 years 17.7 3 
16 -20 years 11.8 2 
20 or more years 35.3 6 
Total 17 
No Responses 1 


Years Responses 

% n 
0-2 years 11.8 2 
3-5 years 29.4 5 
6-10 years 23.5 4 
11-15 years 5.9 1 
16-20 years 17.7 3 
20 or more years 11.8 2 
Total 17 
No Responses 1 


Table A-4. Number of Reading Classes Taught By HISD SRW Respondents, 2017-2018 


No. of Reading Classes 5 Responses 

a n 
1to2 35.3 6 
aa 29.4 5 
5 or more 11.8 2 
Other (please specify) 23.5 A 
Total 17 
No Responses 1 
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Table A-5. Respondents’ Participation in SRW Professional Development Training, 2015-2016 to 


2017-2018 
Responses 

Professional Development No Yes Total Weighted 

% n % n n Average 
The information | learned during the one-day "job-alike" training prepared 56.3 9 438 7 16 1.44 
me to begin the implementation of SRW. 
| attended the "job-alike” for the 2015-2016 school year. 42.9 6 57.1 8 14 1.57 
| attended the "job-alike" for the 2016-2017 school year. 33.3 5 66.7 10 15 1.67 
| attended the "job-alike" for the 2017-2018 school year. 26.7 4 73.3. 11 15 1.73 
Other (please specify) 2 
Total 16 
No Responses 2 


Table A-6. Respondents’ Participation in Monthly SRW Professional Development Training, 


2015-2016 to 2017-2018 


Responses 
Professional Development No Yes Total 
% n % n n 
| attended SRW monthly meetings during the 
2015-2016 school year. ale . ie a — 
| attended SRW monthly meetings during the 
2016-2017 school year. A ‘: fE:6 it an 
| attended the SRW meeting on September 12, 42.9 6 57.1 8 1A 
2017 on establishing classroom routines. 
| attended the SRW meeting on October 12, 2017 35.7 5 64.3 9 re 


on dos and don't of small group instruction. 

| attended the SRW meeting on November 9, 
2017 on reading conference - progress monitoring 28.6 4 71.4 10 14 
with use of authentic "Just Right" texts. 

| attended the SRW meeting on January 11, 2018 


on Vocabulary - Word Works. one 2 cae a0 A 
| attended the SRW meeting on February 8, 2018 

on Authentic Assessment and Student Choice 28.6 4 71.4 10 14 
(Project-Based Learning) 

Total 14 
No Responses 4 


Table A-7. SRW Respondents’ Efficacy and Instructional Practices, 2017-2018 


Responses 
- 5 e Strongly Disagree Somewhat Agree Strongly Total Weighted 
Teacher Efficacy and Instructional Practices Disagree Agree Agree Average 
% n % n % n % n % n n 


n comparison to one year ago, | have a deeper 
understanding of what it means to support student in 0.0 ie} 14.3 2 35.7 5 21.4 3 28.6 4 14 3.64 
need of reading and writing intervention. 

have access to Saddleback "Hi-Low’ texts on my 


23.1 3 23.1 3 30.8 4 7.7 1 15.4 2 13 2.69 
campus. 
The Saddleback "Hi-Low" texts available to my 
classroom are of high quality. 15.4 2 23.1 3 30.8 4 7.7 1 23.1 3 13 3.00 
have access to classroom libraries in all core 30.8 4 77 1 0.0 0 23.1 3 38.5 5 13 3.31 
content-based areas. 
My students work independently on reading and 
writing tasks at their independent level. _ Q af. 1 23-4 3 38.5 2 30.8 # 18 3.92 
meet with my students in small groups around texts 0.0 0 17 1 0.0 0 38.5 5 53.9 7 13 4.38 
at their instructional levels. 
check my students' understanding and give “just-in- 
time" scaffolds. 0.0 0 0.0 0 7.7 1 38.5 5 53.9 7 13 4.46 
monitor my students’ reading and writing 
performance. 0.0 0 0.0 0 0.0 0 23.1 3 76.9 10 13 4.77 
fa my students into need-based intervention 0.0 0 0.0 0 0.0 0 38.5 5 61.5 8 13 4.62 
Total 14 
NO Responses 4 
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Table A-8. SRW Respondents’ Assess i nts’ Reading Skills, 2017-2018 
Responses 
' , ‘ Strongly Disagree Somewhat Agree Strongly Total Weighted 
Students' Reading Skills Disagree Agree Agree Average 
% n % n % n % n % n n 
My students can self-monitor their reading and make adjustments 
using rereading and questioning. 7.7 1 15.4 2 38.5 5 15.4 2 23.1 3 13 3.31 
My students use prior knowledge and experience to comprehend 0.0 0 0.0 0 53.9 7 15.4 2 30.8 4 13 3.77 
texts. E F £ | i 
My students make inferences such as drawing conclusion and making 
generalizations and predictions. 0.0 0 0.0 0 46.2 6 46.2 6 7.7 1 13 3.62 
My students can establish purposes for reading selected texts based 0.0 0 16.7 2 33.3 4 25.0 3 25.0 3 2 3.58 
on their desire to enhance comprehension. . . ° . . 
My students can summarize, paraphrase and synthesize texts in ways 
that make meaning and logical order. 7.7 1 7.7 1 46.2 6 23.1 3 15.4 2 13 3.31 
My students can reflect understanding to monitor their reading 
comprehension. 7.7 1 0.0 0 61.5 8 15.4 2 15.4 2 13 3.31 
My students can monitor and adjust comprehension using 
background knowledge, rereading aloud, and visualizing. fi t ka ° aoe . se 4 ao a i one 
| schedule and confer with my students to provide feedback about 
their reading. 0.0 0 0.0 0 38.5 5 23.1 3 38.5 5 13 4.00 
Total 13 
No Responses 5 


Table A-9. SRW Respondents’ Instructional Writing Practices, 2017-2018 


Responses 

® a . Strongly Disagree Somewhat Agree Strongly Total Weighted 

Instructional Writing Practices Disagree Agree Agree Average 
% n % n % n % n % n n 

My students’ writing tasks reflect a variety of purposes. 0.0 0 7.7 1 38.5 5 46.2 6 7.7 1 13 3.54 
| model the writing process for my students. 0.0 0 0.0 0 7.7 ue 53.9 7 38.5 5 13 4.31 
| support my students through small group instruction and 
collaborative writing experiences to meet their needs. 9.0 0 9.0 0 a 7 46-2 6 isis 18 ane 
| use PLC time to analyze my students’ writing samples. 8.3 1 8.3 1 33.3 4 33.3 4 16.7 2 12 3.42 
; Bint | confer with my students to provide feedback about 0.0 0 83 1 16.7 2 16.7 2 58.3 7 2 4.25 
My students are leading their own writing conferences. 7.7 1 23.1 3 38.5 5 23.1 3 7.7 1 13 3.00 
My students are selecting and publishing pieces to work in their 
writing portfolios. 83 10 167 20 500 6.0 0.0 0.0 25.0 3.0 12.0 3.17 
Total 13 
No Responses = 


Table A-10. Use Frequency of Digital Learning Resources Among SRW Survey Respondents, 


2017-2018 


at , Never Rarely Sometimes Often Very Often Total Weighted 
Use of Digital Learning Resources % n % A % 7 % n % a n Average 
Vocabulary.com 7.1 1 143 2 21.4 3 7.1 150.0 7 14 3.21 
Achieve 3000 0.0 0 143 2 14.3 2. 143 2 57.1 8 14 3.43 
Mindplay 92.3 12 0.0 0 7.7 1 0.0 0 0.0 0 13 1.15 
SRW Hub Master Course 30.8 4 0.0 0 23.1 3. 154 2 308 4 13 2.69 
Digital Magazines 42.9 6 14.3 2 21.4 3 7.1 1 14.3 2 14 2.14 
Imagine Learning 46.2 6 cava 1 15.4 2 7.7 1 23.1 3 13 2.23 
Renaissance Universal Screener 0.0 ie} 7.1 1 21.4 3 50.0 7 21.4 3 14 3.14 
MackinVIA 84.6 11 7.7 1 7.7 1 0.0 0 0.0 0 13 1.23 
Other (please specify) 6 
Total 14 
No Response 4 
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Appendix B 


Table B-1. Paired T-Test of SRW Sixth-Grade Reading, 2017-2018 


STAAR Reading Scale Score n Mean Standard Error Standard Deviation 95% Conf. Interval 


2018 732 1504.5 4.9 131.9 1494.9 1514.0 
2017 732 = =1455.1 2.8 75.6 1449.6 1460.6 
Mean Difference 732 49.4 5.8 157.2 37.9 60.8 
t=8.5 
df = 731 
p. < .001; Effect size = 0.31 
Note: Effect size is Cohen’s d where small effect = 0.2, medium effect = 0.5, and large effect = 0.8. 


2017 STAAR 3-8 reading standard: Did Not Meet = 21464. 
2018 STAAR 3-8 reading standards: Approaches = 1527-1616; Met = 1629-1692; Masters = 1718-2056. 


Table B-2. Paired T-Test of SRW Seventh-Grade Reading, 2017-2018 


STAAR Reading Scale Score n Mean Standard Error Standard Deviation —_ 95% Conf. Interval 
2018 744 1578.3 4.7 127.4 1569.1 1587.5 
2017 744) = 1428.1 2.9 78.6 1422.4 1433.8 
Mean Difference 744 150.2 5.6 153.4 139.1 161.2 
t= 26.7 
df = 743 
P 
p. < .001; Effect size = 0.98 
Note: Effect size is Cohen’s d where small effect = 0.2, medium effect = 0.5, and large effect = 0.8. 


2017 STAAR 3-8 reading standard: Did Not Meet = 21506. 
2018 STAAR 3-8 reading standards: Approaches = 1567-1662; Meets = 1674-1728; Masters = 1753-2142. 


Table B-3. Paired T-Test of SRW Eighth-Grade Reading, 2017-2018 


STAAR Reading Scale Score n Mean Standard Error Standard Deviation 95% Conf. Interval 

2018 577 1623.3 5.6 134.4 1612.4 1634.3 
2017 577 =1434.6 3.1 73.4 1428.6 1440.7 
Mean Difference 577 188.7 6.5 157.3 175.8 201.6 

t= 28.8 

df = 576 

p. < .001; Effect size = 1.20 

Note: Effect size is Cohen’s d where small effect = 0.2, medium effect = 0.5, and large effect = 0.8. 


2017 STAAR 3-8 reading standard: Did Not Meet = 21552. 
2018 STAAR 3-8 reading standards: Approaches = 1580-1691; Met = 1700-1759; Masters = 1783-2141. 
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Table B-4. Percentage of SRW Students by Grade and Selected Attributes who Performed at or 


Above the Approaches Grade Level Standard on the 2018 STAAR 3-8 Reading Tests 


Sixth Grade Seventh Grade Eighth Grade 
Attributes 
n n= 731 n n= 743 n n=577 

Female 357 48.8 467 
Male 281 38.4 
Economically No 451 
Disadvantaged Yes 292 

No 639 
At-Risk 

Yes 243 33.3 349 

. . No 357 48.8 418 

Special Education 

Yes 42 5.8 64 
Limited English No 401 451 
Proficiency Yes 196 26.8 282 37.9 167 29.0 

: (e) 260 35.6 367 49.4 309 53.5 

Gifted and Talented 

Yes 579 646 

Asian 292 40.0 464 

_ Black 342 46.8 379 

Ethnicity : ; 

Hispanic 305 41.7 407 

White 243 33.3 247 


Note: Performance at or above 50 percent who met or exceeded the Approaches Grade Level standard are shaded green, and less 
than 50 percent are shaded in pink. 
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